Uncalibrated Vision-Based Control and Motion Planning of Robotic Arms in Unstructured Environments

Many robotic systems are required to operate in unstructured environments. This imposes significant challenges on algorithm design. Particularly, motion control and planning algorithms should be robust to noise and outliers, because uncertainties are inevitable. In addition, independence from scene model and calibration parameters is preferred; otherwise, the tedious model extraction and calibration procedures need to be redone with every change in the environment. The basic problem that this thesis addresses is how to robustly control the motion of a vision-based manipulator and plan occlusion-free paths in unstructured environments. Vision-based motion control without using calibration or a geometric model is studied in Uncalibrated Visual Servoing (UVS). In this thesis, we adopt a framework based on UVS and contribute to two distinct areas: robust visual servoing and robust randomized path planning. We develop a statistically robust algorithm for UVS, which detects outliers and finds robust estimates of the uncalibrated visual-motor Jacobian, a central matrix in the visual servoing control law. We integrate the robust Jacobian estimation into a real-time feedback control loop and present case studies. To avoid the visual and joint-limit constraints, we propose a robust sampling-based path planning algorithm. The proposed planner fits well within the UVS framework and facilitates occlusion-free paths, despite not knowing the obstacle model. Finally, our third and last contribution is a novel UVS approach based on extracting the geometry of three images in the form of the trifocal tensor. We experimentally validate this approach and show that the proposed UVS controller handles some of the most challenging degenerate configurations of image-based visual servoing.